Web Survey Bibliography
Title Survey Mode and Mail Method: A Practical Experiment in Survey Fielding for a Multi-round Survey
Author Sullivan, B. D.; Duda, N.; Bogen, K.; Clusen, N. A.; Wakar, B.; Zhou, H.
Year 2016
Access date 09.06.2016
Abstract
Researchers can choose from several survey mode and contact method options when designing survey fielding plans. Each of these options is ass
ociated with known costs, yet the expected success of each fielding option, which often depends on the circumstances and preferences of a specific population of respondents, is much more ambiguous. Surveys with multiple rounds present an opportunity for researchers to identify the most successful and cost-efficient survey fielding plan in one round, to guide survey fielding in subsequent rounds. During the fielding of the 2015 Comprehensive Primary Care (CPC)Practice Survey, sponsored by the Centers for Medicare & Medicaid Services, we implemented a two-by-two survey experiment with a subset of our sample of practice managers for primary care practice sitesfor whom we did not have valid email addresses, only practice mailing addresses. We randomly assigned 269 practice managers to receive a letter that directed the practice to either (1) complete the surveyonline with a personalized web login, or (2) complete an enclosed hard copy questionnaire. We also randomly assigned the practice managers in each of those groups to be sent all outreach material using either (1) regular USPS mail or (2) FedEx next-day shipping. Though practice managers in the online and FedEx groups responded to the survey more quickly initially, over time the difference in cumulative response rates between groups disappeared. In further analyses of this experiment, we will examine the impact of survey mode (online versus hard copy) and mail method (USPS versus FedEx) on data quality, representativeness, and cost. We will recommend the best mode and procedures for future rounds of the CPC Practice Survey, as well as reflect on the general lessons learned on conducting fielding experiments in multi-round surveys.
ociated with known costs, yet the expected success of each fielding option, which often depends on the circumstances and preferences of a specific population of respondents, is much more ambiguous. Surveys with multiple rounds present an opportunity for researchers to identify the most successful and cost-efficient survey fielding plan in one round, to guide survey fielding in subsequent rounds. During the fielding of the 2015 Comprehensive Primary Care (CPC)Practice Survey, sponsored by the Centers for Medicare & Medicaid Services, we implemented a two-by-two survey experiment with a subset of our sample of practice managers for primary care practice sitesfor whom we did not have valid email addresses, only practice mailing addresses. We randomly assigned 269 practice managers to receive a letter that directed the practice to either (1) complete the surveyonline with a personalized web login, or (2) complete an enclosed hard copy questionnaire. We also randomly assigned the practice managers in each of those groups to be sent all outreach material using either (1) regular USPS mail or (2) FedEx next-day shipping. Though practice managers in the online and FedEx groups responded to the survey more quickly initially, over time the difference in cumulative response rates between groups disappeared. In further analyses of this experiment, we will examine the impact of survey mode (online versus hard copy) and mail method (USPS versus FedEx) on data quality, representativeness, and cost. We will recommend the best mode and procedures for future rounds of the CPC Practice Survey, as well as reflect on the general lessons learned on conducting fielding experiments in multi-round surveys.
Access/Direct link Conference Homepage (abstract)
Year of publication2016
Bibliographic typeConferences, workshops, tutorials, presentations
Web survey bibliography (290)
- Mind the Mode: Differences in Paper vs. Web-Based Survey Modes Among Women With Cancer; 2017; Hagan, T. L.; Belcher, S. M.; Donovan, H. S.
- Telephone versus Online Survey Modes for Election Studies: Comparing Canadian Public Opinion and Vote...; 2017; Breton, C.; Cutler, F.; Lachance, S.; Mierke-Zatwarnicki, A.
- Paradata as an aide to questionnaire design: Improving quality and reducing burden; 2017; Timm, E.; Stewart, J.; Sidney, I.
- Fieldwork monitoring and managing with time-related paradata; 2017; Vandenplas, C.
- Where, When, How and with What Do Panel Interviews Take Place and Is the Quality of Answers Affected...; 2017; Niebruegge, S.
- Do distractions during web survey completion affect data quality? Findings from a laboratory experiment...; 2017; Wenz, A.
- A Comparison of Emerging Pretesting Methods for Evaluating “Modern” Surveys; 2017; Geisen, E., Murphy, J.
- Pushing to web in the ISSP; 2017; Jonsdottir, G. A.; Dofradottir, A. G.; Einarsson, H. B.
- Push2web or less is more? Experimental evidence from a mixed-mode population survey at the community...; 2017; Neumann, R.; Haeder, M.; Brust, O.; Dittrich, E.; von Hermanni, H.
- Rates, Delays, and Completeness of General Practitioners’ Responses to a Postal Versus Web-Based...; 2017; Sebo, P.; Maisonneuve, H.; Cerutti, B.; Pascal Fournier, J.; Haller, D. M.
- Comparing data quality and cost from three modes of on-board transit surveys ; 2017; Agrawal, A. W.; Granger-Bevan, S.; W.; Newmark, G. L.; Nixon, H.
- Overview: Online Surveys; 2017; Vehovar, V.; Lozar Manfreda, K.
- FocusVision 2015 Annual MR Technology Report; 2016; Macer, T., Wilson, S.
- Comparisons of Online Recruitment Strategies for Convenience Samples: Craigslist, Google AdWords, Facebook...; 2016; Antoun, C., Zhang, C., Conrad, F. G., Schober, M. F.
- Feature phones no barrier to conducting an effective conjoint study ; 2016; de Rooij, R.; Dossin, R.
- Mixing Modes: Challenges (and Tradeoffs) of Adapting a Mailed Paper Survey to the Web ; 2016; Wilkinson-Flicker, S.; McPhee, C. B.; Medway, R.; Kaiser, A.; Cutts, K.
- Does Changing Monetary Incentive Schemes in Panel Studies Affect Cooperation? A Quasi-experiment on...; 2016; Schaurer, I.; Bosnjak, M.
- Survey Mode and Mail Method: A Practical Experiment in Survey Fielding for a Multi-round Survey ; 2016; Sullivan, B. D.; Duda, N.; Bogen, K.; Clusen, N. A.; Wakar, B.; Zhou, H.
- Early-bird Incentives: Results From an Experiment to Determine Response Rate and Cost Effects ; 2016; De Santis, J.; Callahan, R.; Marsh, S.; Perez-Johnson, I.
- Using Cash Incentives to Help Recruitment in a Probability Based Web Panel: The Effects on Sign Up Rates...; 2016; Krieger, U.
- Assessing Changes in Coverage Bias of Web Surveys a s Internet Access Increases in the United States...; 2016; Sterrett, D.; Malato, D.; Benz, J.; Tompson, T.; English, N.
- Thinking Inside the Box Visual Design of the Response Box Affects Creative Divergent Thinking in an...; 2016; Mohr, A. H.; Sell, A.; Lindsay, T.
- Web surveys for offline rural communities ; 2016; Gichohi, B. W.
- Adaptive survey designs to minimize survey mode effects – a case study on the Dutch Labor Force...; 2016; Calinescu, M.; Schouten, B.
- Assessment of Innovations in Data Collection Technology for Understanding Society; 2016; Couper, M. P.
- Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk; 2016; Berinsky, A.; Huber, G. A.; Lenz, G. S.
- Sample Representation and Substantive Outcomes Using Web With and Without Incentives Compared to Telephone...; 2016; Lipps, O.; Pekari, N.
- “Money Will Solve the Problem”: Testing the Effectiveness of Conditional Incentives for...; 2016; DeCamp, W.; Manierre, M. J.
- Effects of Incentive Amount and Type of Web Survey Response Rates; 2016; Coopersmith, J.; Vogel, L. K.; Bruursema, T.; Feeney, K.
- Effect of a Post-paid Incentive on Response to a Web-based Survey; 2016; Brown, J. A.; Serrato, C. A.; Hugh, M.; Kanter, M. H.; A.; Spritzer, K. L.; Hays, R. D.
- Take the money and run? Redemption of a gift card incentive in a clinician survey. ; 2016; Chen, J. S.; Sprague, B. L.; Klabunde, C. N.; Tosteson, A. N. A.; Bitton, A.; Onega, T.; MacLean, C....
- A reliability analysis of Mechanical Turk data; 2016; Rouse, S. V.
- Taming Big Data: Using App Technology to Study Organizational Behavior on Social Media; 2015; Bail, C. A.
- Using Mobile Phones for High-Frequency Data Collection; 2015; Azevedo, J. P.; Ballivian, A.; Durbin, W.
- Cell Phone and Face-to-face Interview Responses in Population-based Sur- veys - How Do They Compare?; 2015; Ghandour, L.; Ghandour, B.; Mahfoud, Z.; Mokdad, A.; Sibai, A. M.
- The Cathie Marsh lecture: What does the failure of the polls tell us about the future of survey research...; 2015; Sturgis, P., Matheson, J.
- Mixed mode surveys ; 2015; Burton, J.
- Understanding Society Innovation Panel Wave 7: Results from Methodological Experiments; 2015; Blom, A. G.; Burton, J.; Booker, C. L.; Cernat, A.; Fairbrother, M.; Jaeckle, A.; Kaminska, O.; Keusch...
- Response Effects of Prenotification, Prepaid Cash, Prepaid Vouchers, and Postpaid Vouchers: An Experimental...; 2015; van Veen, F.; Goeritz, A.; Sattler, S.
- Tailored fieldwork design to increase representative household survey response: an experiment in the...; 2015; Luiten, A.; Schouten, B.
- Using Internet to Recruit Immigrants with Language and Culture Barriers for Tobacco and Alcohol Use...; 2015; Carlini, B. H.; Safioti, L.; Rue, T. C.; Miles, L.
- Online Recruitment Methods for Web-Based and Mobile Health Studies: A Review of the Literature; 2015; Lane, T. S.; Armin, J.; Gordon, Ju. S.
- iTunes Song-Gifting is a Low-Cost, Efficient Recruitment Tool to Engage High-Risk MSM in Internet Research...; 2015; Holland, C. M.; Ritchie, N. D.; Du Bois, S. N.
- A Note on the Use of Survey Research Firms to Enable Empirical Data Collection; 2015; Schoenherr, T.; Ellram, L. M.; Tate, W. L.
- Recruiting Online: Lessons From a Longitudinal Survey of Contraception and Pregnancy Intentions of Young...; 2015; Harris, M. L.; Loxton, D.; Wigginton, B.; Lucke, J. C.
- Recruiting for addiction research via Facebook; 2015; Thornton, L. K.; Harris, K.; Baker, A.; Johnson, M.; Kay-Lambkin, F. J.
- The effectiveness of incentives on recruitment and retention rates: an experiment in a web survey; 2015; Mulder, J.; Douhou, S.
- Adding a Web Mode to Phone Surveys: Effectiveness and Cost Implications; 2015; Beebe, T. J.; Lien, R.; Luxenberg, H.; Rainey, J.
- Query on Data Collection for Social Surveys; 2014; Blanke, K., Luiten, A.
- Examining the Effect of Prenotification Postcards on Online Survey Response Rate in a University Graduate...; 2014; Lalasz, C. B., Doane, M. J., Springer, V. A., Dahir, V. B.